Icon

[01.CX] Combining Siloed Legacy Data with Running App's Back-End Database

Siloed Legacy Datato theCloud Data Warehouse For Reuse, Repurpose & For Data Governance 1) Context: At the start of this Customer Experience Center, the team used Google Sheets tocapture information since the operation was in its early stages. Later on, with support from acentralized Business Intelligence and Data Analytics team, they designed and developed acomprehensive data ecosystem and reporting system. However, Google Sheets continuedto be used initially to manage information before Digital Initiative team implement a digitaltwin of their operation 2) Context: When the Digital Initiative team implemented an operating system to support theworkflow of this Customer Experience Center, we collaborated to set a cut-off date. This datemarked the point at which legacy data would no longer be used, and instead, the system wouldrely solely on real-time data from the backend database. This decision was made to ensurethat the most current and accurate data was being used to inform decision-making processesand improve overall efficiency. 3) Context: After transforming and ensuring the quality of the legacy data, and obtainingapproval for its integrity, we migrated it to the Snowflake cloud data warehouse in a one-timetransfer. This migration is a crucial part of our digital transformation efforts, aimed at breakingdown silos Live Data from a Running App For Analytics & For Data Governance 6) Context: A senior management member has made a last-minute request tomodify the naming on the dashboard, which serves as the front-end for all thebacked-end processes. This approach is not recommended as it is not future-proof.To address this, I need to collaborate with the backend team. I believe this presentsan opportunity for business-side individuals to enhance their understanding ofsystem designs and engineering. By doing so, we can foster better collaborationbetween system and business personnel and work towards a common goal. 4) Context: This node plays a crucial role in my workflow as it separates legacy data at aspecific point in time and combines it with real-time data collected after the cut-off date. 5) Context: Although the Digital Initiative team developed adigital twin of the customer experience center operation,any changes made to the operation can affect the datastructure, naming conventions, and data integration of theapplication. Consequently, a robust system must be rebuiltto handle this messy data in real-time. This temporaryapproach has been implemented to address this issue. AuthenticatedGet AccessRestructured naming convention of OLD legacy systems to align withCURRENT naming convention across ALL the reporting & operation systems [Cut-off Date]118 Records are excludedCleaned Data for PBIDashboardAgent ListLegacy Hotline Data for ANOTHER Reporting PipelineCX Center Calls & InquiriesBPMS LiveDatabaseConnectionBPMS Live Cleaning Approach(Static=Need to Update)Error CheckLast-minute requeston Naming StructureGoogleAuthentication Google SheetsConnection Google SheetsReader String to Date&Time Rule-basedRow Splitter Excel Writer Snowflake Connector Table Creator Column Filter DB Query Reader MySQL Connector Excel Reader Joiner Rule-basedRow Filter Column Rename ReferenceRow Filter Concatenate Data Transformation DB Update DB Loader Row Splitter Joiner Concatenate Rule Engine Data Cleaning &Transformation Data Cleaning &Transformation Data Cleaning &Transformation FilteringUnnecessary Data Enriching Data Siloed Legacy Datato theCloud Data Warehouse For Reuse, Repurpose & For Data Governance 1) Context: At the start of this Customer Experience Center, the team used Google Sheets tocapture information since the operation was in its early stages. Later on, with support from acentralized Business Intelligence and Data Analytics team, they designed and developed acomprehensive data ecosystem and reporting system. However, Google Sheets continuedto be used initially to manage information before Digital Initiative team implement a digitaltwin of their operation 2) Context: When the Digital Initiative team implemented an operating system to support theworkflow of this Customer Experience Center, we collaborated to set a cut-off date. This datemarked the point at which legacy data would no longer be used, and instead, the system wouldrely solely on real-time data from the backend database. This decision was made to ensurethat the most current and accurate data was being used to inform decision-making processesand improve overall efficiency. 3) Context: After transforming and ensuring the quality of the legacy data, and obtainingapproval for its integrity, we migrated it to the Snowflake cloud data warehouse in a one-timetransfer. This migration is a crucial part of our digital transformation efforts, aimed at breakingdown silos Live Data from a Running App For Analytics & For Data Governance 6) Context: A senior management member has made a last-minute request tomodify the naming on the dashboard, which serves as the front-end for all thebacked-end processes. This approach is not recommended as it is not future-proof.To address this, I need to collaborate with the backend team. I believe this presentsan opportunity for business-side individuals to enhance their understanding ofsystem designs and engineering. By doing so, we can foster better collaborationbetween system and business personnel and work towards a common goal. 4) Context: This node plays a crucial role in my workflow as it separates legacy data at aspecific point in time and combines it with real-time data collected after the cut-off date. 5) Context: Although the Digital Initiative team developed adigital twin of the customer experience center operation,any changes made to the operation can affect the datastructure, naming conventions, and data integration of theapplication. Consequently, a robust system must be rebuiltto handle this messy data in real-time. This temporaryapproach has been implemented to address this issue. AuthenticatedGet AccessRestructured naming convention of OLD legacy systems to align withCURRENT naming convention across ALL the reporting & operation systems [Cut-off Date]118 Records are excludedCleaned Data for PBIDashboardAgent ListLegacy Hotline Data for ANOTHER Reporting PipelineCX Center Calls & InquiriesBPMS LiveDatabaseConnectionBPMS Live Cleaning Approach(Static=Need to Update)Error CheckLast-minute requeston Naming StructureGoogleAuthentication Google SheetsConnection Google SheetsReader String to Date&Time Rule-basedRow Splitter Excel Writer Snowflake Connector Table Creator Column Filter DB Query Reader MySQL Connector Excel Reader Joiner Rule-basedRow Filter Column Rename ReferenceRow Filter Concatenate Data Transformation DB Update DB Loader Row Splitter Joiner Concatenate Rule Engine Data Cleaning &Transformation Data Cleaning &Transformation Data Cleaning &Transformation FilteringUnnecessary Data Enriching Data

Nodes

Extensions

Links

Download

To use this workflow in KNIME, download it from the below URL and open it in KNIME:

Download Workflow
Created by: Data Analytics Team
Created at: 2023-04-16
On NodePit since: 2023-04-16
Last update: 2024-11-17
Created with KNIME version: v4.5.2

Deploy, schedule, execute, and monitor your KNIME workflows locally, in the cloud or on-premises – with our brand new NodePit Runner.

Try NodePit Runner!